Papers with Full-Parameter Fine-Tuning
Look Within or Beyond? A Theoretical Comparison Between Parameter-Efficient and Full Fine-Tuning (2026.acl-long)
Copied to clipboard
| Challenge: | Parameter-Efficient Fine-Tuning (PEFT) is an alternative to Full-Parameter Fine-tuning, but its effectiveness on complex tasks such as reasoning and instruction-following remains unclear. |
| Approach: | They propose to use PEFT to reduce the number of trainable parameters while freezing the weights of LLMs. |
| Outcome: | The proposed methods perform well on standard tasks, but weaknesses on complex and adversarial settings call for new directions beyond current paradigms. |